U.S. patent application number 12/669685 was filed with the patent office on 2010-11-11 for multiple resolution video network with eye tracking based control.
Invention is credited to Yang Cai.
Application Number | 20100283843 12/669685 |
Document ID | / |
Family ID | 40260083 |
Filed Date | 2010-11-11 |
United States Patent
Application |
20100283843 |
Kind Code |
A1 |
Cai; Yang |
November 11, 2010 |
MULTIPLE RESOLUTION VIDEO NETWORK WITH EYE TRACKING BASED
CONTROL
Abstract
A system. The system includes a computing device configured for
communication with a plurality of multiple resolution cameras and
with a display device. The computing device includes a camera
resolution module configured for instructing at least one of the
multiple resolution cameras to operate at a first resolution at a
first period of time and at a second resolution at a second period
of time. The first resolution is different than the second
resolution.
Inventors: |
Cai; Yang; (Pittsburgh,
PA) |
Correspondence
Address: |
REED SMITH LLP
P.O. BOX 488
PITTSBURGH
PA
15230-0488
US
|
Family ID: |
40260083 |
Appl. No.: |
12/669685 |
Filed: |
July 17, 2008 |
PCT Filed: |
July 17, 2008 |
PCT NO: |
PCT/US08/70375 |
371 Date: |
May 12, 2010 |
Related U.S. Patent Documents
|
|
|
|
|
|
Application
Number |
Filing Date |
Patent Number |
|
|
60959821 |
Jul 17, 2007 |
|
|
|
60959820 |
Jul 17, 2007 |
|
|
|
Current U.S.
Class: |
348/78 ;
348/207.1; 348/E5.024; 348/E7.091 |
Current CPC
Class: |
H04N 5/23293 20130101;
H04N 7/181 20130101; H04N 5/23245 20130101; G06K 9/00228 20130101;
H04N 7/12 20130101; H04N 5/247 20130101; H04N 5/23206 20130101 |
Class at
Publication: |
348/78 ;
348/207.1; 348/E05.024; 348/E07.091 |
International
Class: |
H04N 7/18 20060101
H04N007/18; H04N 5/225 20060101 H04N005/225 |
Goverment Interests
STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH
[0002] This invention was made with United States Government
support in the form of Grant No. DAAD19-02-1-0389 from the Army
Research Office. The United States Government may have certain
rights in the invention.
Claims
1. A system, comprising: a computing device configured for
communication with a plurality of multiple resolution cameras and
with a display device, wherein the computing device comprises: a
camera resolution module configured for instructing at least one of
the multiple resolution cameras to operate at: a first resolution
at a first period of time; and a second resolution at a second
period of time, wherein the first resolution is different than the
second resolution.
2. The system of claim 1, wherein the computing device further
comprises an eye tracking module in communication with the camera
resolution module, wherein the eye tracking module is configured
for associating a first position of a person's eye with a first
location on the display device.
3. The system of claim 1, wherein the computing device further
comprises an eye tracking module in communication with the camera
resolution module, wherein the eye tracking module is configured
for associating a first position of a person's eye with a first
image on the display device.
4. The system of claim 1, wherein the computing device further
comprises an eye tracking module in communication with the camera
resolution module, wherein the eye tracking module is configured
for associating a first position of a person's eye with a first one
of the multiple resolution cameras.
5. The system of claim 1, wherein the computing device further
comprises an eye tracking module in communication with the camera
resolution module, wherein the eye tracking module is configured
for associating a first location on the display device with a first
image on the display device.
6. The system of claim 1, wherein the computing device further
comprises an eye tracking module in communication with the camera
resolution module, wherein the eye tracking module is configured
for associating a first location on the display device with a first
one of the multiple resolution cameras.
7. The system of claim 1, wherein the computing device further
comprises an eye tracking module in communication with the camera
resolution module, wherein the eye tracking module is configured
for associating a first image on the display device with a first
one of the multiple resolution cameras.
8. The system of claim 1, wherein the computing device further
comprises an eye tracking module in communication with the camera
resolution module, wherein the camera resolution module is further
configured for instructing the at least one of the multiple
resolution cameras based on information received from the eye
tracking module.
9. The system of claim 1, wherein the computing device further
comprises a display module configured for sending a plurality of
images to the display device, wherein the plurality of images
comprise: a first image at the first resolution; and a second image
at the second resolution.
10. A method, implemented at least in part by a computing device,
the method comprising: receiving a first image from a multiple
resolution camera at a first resolution; generating a change of
resolution instruction; sending the change of resolution
instruction to the multiple resolution camera; and receiving a
second image from the multiple resolution camera at a second
resolution, wherein the second resolution is different than the
first resolution,
11. The method of claim 10, wherein generating the change of
resolution instruction comprises generating the change of
resolution instruction based on a position of a person's eye.
12. The method of claim 10, wherein generating the change of
resolution instruction comprises generating the change of
resolution instruction after a position of a person's eye has
remained fixed for a predetermined period of time.
13. The method of claim 10, further comprising associating a
position of a person's eyes with a location on a display
device.
14. The method of claim 10, further comprising associating a
position of a person's eyes with an image on a display device.
15. The method of claim 10, further comprising associating a
position of a person's eyes with a multiple resolution camera.
16. The method of claim 10, further comprising associating a
location on a display device with an image on the display
device.
17. The method of claim 10, further comprising associating a
location on a display device with a multiple resolution camera.
18. The method of claim 10, further comprising associating an image
on a display device with a multiple resolution camera.
19. The method of claim 10, further comprising: generating a second
change of resolution instruction; sending the second change of
resolution instruction to the multiple resolution camera; and
receiving a third image from the multiple resolution camera at the
first resolution.
Description
CROSS-REFERENCE TO RELATED APPLICATION
[0001] This application claims the benefit of the earlier filing
date of U.S. Patent Application No. 60/959,820 filed on Jul. 17,
2007 and U.S. Patent Application No. 60/959,821 filed on Jul. 17,
2007. This application is related to the International Application
entitled "Multiple Resolution Video Network With Context Based
Control", filed concurrently herewith.
BACKGROUND
[0003] This application discloses an invention which is related,
generally and in various embodiments, to a multiple resolution
video network with eye tracking based control.
[0004] Video networks are becoming more and more commonplace,
including those utilizing digital cameras in the security and
surveillance fields. For most security and surveillance
applications, more optimal results are generally realized when more
high resolution cameras are included in the video network. More
cameras may be utilized to cover a larger geographic area, increase
the number of views associated with a particular area, decrease the
number of "blind" spots, etc.
[0005] However, in many current video networks, especially those
which include wireless channels, the maximum bandwidth of the video
network often operates to limit the number of high resolution
cameras which can be effectively included in the video network. The
amount of bandwidth generally needed to transmit high resolution
images (e.g., 640.times.480 pixels) from a high resolution camera
at a high frame rate (e.g., 30 frames per second) and at a low
compression rate percentage (e.g., 10%) is on the order of
approximately nine Megabits per second. Thus, for a video network
which includes a wireless channel, the video network may not be
able to support a single high resolution camera, depending on the
desired resolution, frame rate and compression rate percentage. For
example, the approximately nine Megabits per second bandwidth
needed in the above example far exceeds the capacity of current
Bluetooth technology, which is only on the order of approximately
three Megabits per second.
[0006] The capacity problem is not limited to video networks which
include wireless channels. For a video network which includes
twelve high resolution cameras, the required bandwidth generally
needed to concurrently transmit high resolution images (e.g.,
640.times.480 pixels) from the twelve high resolution cameras at a
high frame rate (e.g., 30 frames per second) and at a low
compression rate percentage (e.g., 10%) is on the order of
approximately one-hundred and eight Megabits per second, which
exceeds the capacity of traditional Ethernet cable, which is only
on the order of approximately one-hundred Megabits per second.
Thus, depending on the desired resolution, frame rate and
compression percentage, video networks utilizing traditional
Ethernet cable are often limited to including fewer than twelve
high resolution cameras in the video network.
[0007] Additionally, for video networks which include a plurality
of high resolution cameras, it is generally not practical for a
person or persons to intently view each and every one of the high
resolution images transmitted by the cameras. In general, each
person is typically limited to intently viewing the images from
only one camera at a time. Thus, the scheme of sending all images
at a high resolution, a high frame rate, and a low compression
percentage rate tends to be an inefficient use of valuable network
bandwidth.
SUMMARY
[0008] In one general respect, this application discloses a system.
According to various embodiments, the system includes a computing
device configured for communication with a plurality of multiple
resolution cameras and with a display device. The computing device
includes a camera resolution module configured for instructing at
least one of the multiple resolution cameras to operate at a first
resolution at a first period of time and at a second resolution at
a second period of time. The first resolution is different than the
second resolution.
[0009] In another general respect, this application discloses a
method. The method is implemented at least in part by a computing
device. According to various embodiments, the method includes
receiving a first image from a multiple resolution camera at a
first resolution, generating a change of resolution instruction,
sending the change of resolution instruction to the multiple
resolution camera, and receiving a second image from the multiple
resolution camera at a second resolution. The second resolution is
different than the first resolution.
[0010] Aspects of the invention may be implemented by a computing
device and/or a computer program stored on a computer-readable
medium. The computer-readable medium may comprise a disk, a device,
and/or a propagated signal.
BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Various embodiments of the invention are described herein in
by way of example in conjunction with the following figures,
wherein like reference characters designate the same or similar
elements.
[0012] FIG. 1 illustrates various embodiments of a system;
[0013] FIG. 2 illustrates various embodiments of a display device
of the system of FIG. 1;
[0014] FIG. 3 illustrates various embodiments of a control system
of the system of FIG. 1;
[0015] FIG. 4 illustrates various embodiments of another
system;
[0016] FIG. 5 illustrates various embodiments of a method for
controlling the data flow rate of a video network; and
[0017] FIG. 6 illustrates various embodiments of another method for
controlling the data flow rate of a video network.
DETAILED DESCRIPTION
[0018] It is to be understood that at least some of the figures and
descriptions of the invention have been simplified to illustrate
elements that are relevant for a clear understanding of the
invention, while eliminating, for purposes of clarity, other
elements that those of ordinary skill in the art will appreciate
may also comprise a portion of the invention. However, because such
elements are well known in the art, and because they do not
facilitate a better understanding of the invention, a description
of such elements is not provided herein.
[0019] FIG. 1 illustrates various embodiments of a system 10. The
system 10 includes a plurality of multiple resolution video cameras
12, a control system 14 in communication with the cameras 12, and a
display device 16 in communication with the control system 14 For
purposes of simplicity, only four cameras 12 are shown in FIG. 1.
However, it will be appreciated that the system 10 may include any
number of cameras 12.
[0020] Each multiple resolution video camera 12 is configured for
operation at more than one resolution, and includes a resolution
selection module 18 which is configured to switch the camera 12 to
a given resolution. The cameras 12 may be embodied as any suitable
multiple resolution cameras. For example, the cameras 12 may be
embodied as cameras similar to network cameras manufactured by Axis
Communications AB of Lund, Sweden. As will be explained in more
detail hereinbelow, the resolution of each camera 12 may be
dynamically controlled to switch from one resolution to another.
Thus, a given camera 12 may operate at a first resolution at a
first time period, and at a second resolution at a second time
period. For example, the given camera 12 may operate at a
resolution of 640.times.480 pixels at a first time period, and at a
resolution of 320.times.240 pixels at a second time period.
Although the above example describes the operation of the given
camera 12 in the context of a "high" resolution (640.times.480) and
a "low" resolution (320.times.240), it will be appreciated that the
multiple resolution video cameras 12 may be configured for
operation at resolutions other than 640.times.480 and
320.times.240. In addition, it will be further appreciated that
according to various embodiments, the multiple resolution cameras
12 may be configured for operation at more than two different
resolutions (e.g., at high, medium, and low resolutions).
[0021] The cameras 12 are configured to capture images (i.e.,
frames) at either the first resolution or at the second resolution,
and to send the captured images to the control system 14. The
cameras 12 are also configured to send the captured images to the
control system 14 at any suitable frame rate. For example,
according to various embodiments, the cameras 12 may operate to
send the captured images to the control system 14 at a frame rate
of thirty frames per second. As used herein, the phrase image can
mean a single image (i.e., a single frame) or a plurality of images
(i.e., a plurality of frames). According to other embodiments, as
described in more detail hereinbelow with respect to FIG. 4, the
cameras 12 may also be configured to generate images associated
with the captured images, and to send the associated images to the
control system 14.
[0022] As shown in FIG. 1, according to various embodiments, the
cameras 12 are in communication with the control system 14 via a
network 20. In general, the cameras 12 and the control system 14
each include hardware and/or software components for communicating
with the network 20 and with each other. The cameras 12 and the
control system 14 may be structured and arranged to communicate
through the network 20 via wired and/or wireless pathways using
various communication protocols (e.g., HTTP, TCP/IP, UDP, WAP,
WiFi, Bluetooth) and/or to operate within or in concert with one or
more other communications systems.
[0023] The network 20 may include any type of delivery, system
including, but not limited to, a local area network (e.g.,
Ethernet), a wide area network (e.g. the Internet and/or World Wide
Web), a telephone network (e.g., analog, digital, wired, wireless,
PSTN, ISDN, GSM, GPRS, and/or xDSL), a packet-switched network, a
radio network, a television network, a cable network, a satellite
network, and/or any other wired or wireless communications network
configured to carry data. The network 20 may include elements, such
as, for example, intermediate nodes, proxy servers, routers,
switches, and adapters configured to direct and/or deliver
data.
[0024] The display device 16 may be embodied as any suitable
display device. In general, the display device 16 is configured to
display the images sent by the respective cameras 12, and the
images may be displayed by the display device 16 in any suitable
arrangement. As the respective cameras 12 may operate at more than
one resolution, the display device 16 may display a number of
images at one resolution (e.g., a low resolution), and at least one
other image at a different resolution (e.g., a high resolution).
For example, as shown in FIG. 2, a "high" resolution image may be
displayed on the "left" side of the display device 16 and "low"
resolution images may be displayed on the "right" side of the
display device 16. According to other embodiments, a "high"
resolution image may be displayed proximate a center of the display
device 16 and "low" resolution images may be displayed around or
proximate the "high" resolution image. Thus, it is understood that
the display device 16 may display the images in many other
arrangements.
[0025] As described in more detail hereinbelow with respect to FIG.
4, according to other embodiments, the display device 16 may also
be configured to display a composite image which includes a high
resolution portion and a low resolution portion.
[0026] For purposes of simplicity, only one display device 16 is
shown in FIG. 1. However, it is understood that the system 10 may
include any number of display devices 16. For example, according to
various embodiments, the system 10 may include two display devices
16--one for displaying the "high" resolution image and the other
for displaying the "low" resolution images.
[0027] FIG. 3 illustrates various embodiments of the control system
14 of FIG. 1. The control system 14 includes an eye tracking device
22, and a computing device 24 which is in communication with the
plurality of cameras 12, the display device 16, and the eye
tracking device 22. For purposes of simplicity, only one eye
tracking device 22 and one computing device 24 are shown in FIG. 3.
However, it is understood that the control system 14 may include
any number of eye tracking devices 22 and any number of computing
devices 24. For example, according to various embodiments, the
system 10 may include one eye tracking device 22 for each display
device 16.
[0028] The eye tracking device 22 may be embodied as any suitable
eye tracking device. For example, according to various embodiments,
the eye tracking device 22 may be embodied as or similar to the
EyeTech TM2 model manufactured by EyeTech Digital Systems, Inc. of
Mesa, Arizona. For such embodiments, a first infrared light 22a of
the eye tracking device 22 is positioned proximate a first edge
(e.g., a "left" edge) of the display device 16 and a second
infrared light 22b of the eye tracking device 22 is positioned
proximate a second edge (e.g., a "right" edge) of the display
device 16. (See FIG. 2). In general, the first and second infrared
lights 22a, 22b are utilized to detect and/or track the position of
a person's eyes who is viewing the display device 16.
[0029] The computing device 24 includes a display module 26, an eye
tracking module 28, and a camera resolution module 30. The display
module 26 is in communication with the display device 16, and is
configured for delivering images sent from the cameras 12 to the
display device 16. As each individual camera 12 is configured for
operation at more than one resolution, it is understood that the
display module 26 may deliver images of different resolutions to
the display device 16 at a given time, For example, in a system 10
with four cameras 12, the display module 26 may deliver the images
sent from one of the four cameras 12 to the display device 16 at a
first resolution (e.g., at a "high" resolution) and the respective
images sent from the other three cameras 12 to the display device
16 at a second resolution (e.g., at a "low" resolution).
[0030] The eye tracking module 28 is in communication with the eye
tracking device 22, and is configured for associating an individual
camera 12 with a position of a person's eye (or eyes) who is
viewing the display device 16. In general, when the person focuses
on one of the respective images on the display device 16, the
person's eye will be more focused on that image than on the other
images. According to various embodiments, the eye tracking module
28 associates the position of the person's eye with a position on
the display device 16, associates the position on the display
device 16 with an image on the display device 16, and associates
the image on the display device 16 with an individual camera
12.
[0031] The camera resolution module 30 is in communication with the
plurality of cameras 12, and is configured for dynamically
instructing each camera 12 which resolution to operate at based on
information determined by the eye tracking module 28. Such
information includes which image on the display device 16 the
person's eye is focusing on, and which camera 12 sent the image.
According to various embodiments, when a person's eye is focused on
a particular image on the display device 16, the camera resolution
module 30 instructs the resolution selection module 18 of the
appropriate camera 12 to operate the camera 12 at a high
resolution. For each camera 12 which is not associated with the
particular image, the camera resolution module 30 instructs the
appropriate resolution selection modules 18 to operate the
corresponding cameras 12 at a low resolution. For such embodiments,
the high resolution image sent by the given camera 12 will be
displayed at a high resolution on the display device 16, and the
respective low resolution images sent by the other cameras 12 will
be displayed at a low resolution on the display device 16. The
format of the instruction to change a camera 12 from one resolution
to another resolution may be realized in any suitable manner. For
example, according to various embodiments, the camera resolution
module 30 may send a simple high or low signal (e.g., a "0" or a
"1") to the resolution selection module 18 of a given camera 12 to
initiate a change of the resolution of the camera 12.
[0032] The modules 18, 26, 28, 30 may be implemented in either
hardware, firmware, software or combinations thereof. For
embodiments utilizing software, the software may utilize any
suitable computer language (e.g., C, C++, Java, JavaScript, Visual
Basic, VBScript, Delphi) and may be embodied permanently or
temporarily in any type of machine, component, physical or virtual
equipment, storage medium, or propagated signal capable of
delivering instructions to a device. The respective modules 18
(e.g., software application, computer program) may be stored on
computer-readable mediums of the corresponding cameras 12 such that
when the mediums are read, the functions described herein are
performed. Similarly, the modules 26, 28, 30 (e.g., software
application, computer program) may be stored on another
computer-readable medium (e.g., disk, device, and/or propagated
signal) such that when a computer reads the medium, the functions
described herein are performed.
[0033] According to various embodiments, the respective modules 18
may reside at the corresponding cameras 12. Each of the modules 26,
28, 30 may be in communication with one another, and may reside at
the computing device 24, at other devices within the system 10, or
combinations thereof. For embodiments where the system 10 includes
more than one computing device 24, the modules 26, 28, 30 may be
distributed across a plurality of computing devices 24. According
to various embodiments, the functionality of the modules 26, 28, 30
may be combined into fewer modules (e.g., a single module).
[0034] FIG. 4 illustrates various embodiments of another system 40.
The system 40 is similar to the system 10 of FIG. 1, but is
different in the ways described hereinbelow. In the system 40 of
FIG. 4, each of the cameras 12 further includes an image resolution
module 42. According to various embodiments, each of the image
resolution modules 42 is configured to determine whether a high
resolution image captured by the corresponding camera 12 includes a
particular object of interest. The respective image resolution
modules 42 may be configured to determine whether a variety of
different objects of interest are included in a given high
resolution image. For example, according to various embodiments, a
human face may be an object of interest. The determination may be
realized in any suitable manner. For example, according to various
embodiments, facial recognition software (e.g., software
functionally similar to Intel Open CV) residing at the image
resolution module 42 may be utilized to determine whether a high
resolution image captured by the corresponding camera 12 includes a
human face.
[0035] Each of the image resolution modules 42 is also configured
to define a location of the object of interest within the high
resolution image captured by the corresponding camera 12 when the
image resolution module 42 determines that the high resolution
image includes an object of interest. The location of the object of
interest relative to the entire high resolution image captured by
the corresponding camera 12 may be defined in any suitable manner.
For example, according to various embodiments, the relative
location of the object of interest is defined by coordinates (e.g.,
the four corners of the object of interest, the center point and
radius of the object of interest, etc.) associated with the object
of interest.
[0036] Each of the image resolution modules 42 is further
configured to generate two images associated with the high
resolution image captured by the corresponding camera 12 when the
image resolution module 42 determines that the high resolution
image includes an object of interest. The first associated image is
a high resolution image of the object of interest portion (e.g.,
the portion defined by the coordinates) of the high resolution
image captured by the camera 12. The second associated image is a
low resolution image of the high resolution image captured by the
camera 12. According to various embodiments, the location of the
object of interest relative to the entire high resolution image
captured by the corresponding camera 12, and the two associated
images generated by a given image resolution module 42, are sent to
the control system 14 in lieu of the high resolution image captured
by the corresponding camera 12. Collectively, the two associated
images and the relative location of the object of interest may be
considered to be composite information.
[0037] The system 40 of FIG. 4 is also different from the system 10
of FIG. 1 in that the computing device 24 of the control system 14
of system 40 further includes a composite image module 44.
According to various embodiments, the composite image module 44 is
configured to generate a composite image based on the composite
information sent from a given camera 12. The composite image module
44 is in communication with the display module 26, and is
configured to send generated composite images to the display module
26. The system 40 of FIG. 4 is also different from the system 10 of
FIG. 1 in that the display module 26 is further configured to send
a composite image generated by the composite image module 44 to the
display device 16.
[0038] The composite image module 44 may generate a composite image
in any suitable manner. For example, according to various
embodiments, the composite image module 44 generates the composite
image by superimposing the first associated image (i.e., the high
resolution image of the object of interest portion of the high
resolution image captured by the camera 12) on the second
associated image (i.e., the low resolution image of the high
resolution image captured by the camera 12) at the location
determined by the image resolution module 42.
[0039] According to other embodiments, the composite image module
44 generates the composite image by deleting a portion of the
second associated image corresponding to the location of the object
of interest as determined by the image resolution module 42. The
composite image module 44 then inserts the first associated image
onto the remaining portion of the second associated image at the
location previously occupied by the deleted portion of the second
associated image.
[0040] According to yet other embodiments, the composite image
module 44 generates the composite image by deleting a portion of
the second associated image corresponding to the location of the
object of interest as determined by the image resolution module 42.
The composite image module 44 then positions the remaining portion
of the second associated image over the first associated image such
that the location previously occupied by the deleted portion of the
second associated image is aligned with the first associated
image.
[0041] The modules 42, 44 may be implemented in either hardware,
firmware, software or combinations thereof. For embodiments
utilizing software, the software may utilize any suitable computer
language (e.g., C, C++, Java, JavaScript, Visual Basic, VBScript,
Delphi) and may be embodied permanently or temporarily in any type
of machine, component, physical or virtual equipment, storage
medium, or propagated signal capable of delivering instructions to
a device. The respective modules 42 (e.g., software application,
computer program) may be stored on computer-readable mediums of the
corresponding cameras 12 such that when the mediums are read, the
functions described herein are performed. Similarly, the module 44
(e.g., software application, computer program) may be stored on
another computer-readable medium (e.g., disk, device, and/or
propagated signal) such that when a computer reads the medium, the
functions described herein are performed.
[0042] According to various embodiments, the respective modules 42
may reside at the corresponding cameras 12, and may be in
communication with the corresponding resolution selection modules
18. The module 44 may be in communication with the modules 26, 28
and 30, and may reside at the computing device 24, at other devices
within the system 40, or combinations thereof. For embodiments
where the system 40 includes more than one computing device 24, the
module 40 may be distributed across a plurality of computing
devices 24.
[0043] FIG. 5 illustrates various embodiments of a method 50 for
controlling the data flow rate of a video network. The method 50
may be implemented by various components of the system 10 of FIG.
1. For purposes of simplicity, the method 50 will be described in
the context of the system 10 of FIG. 1.
[0044] Prior to the start of the process 50, each of the cameras 12
may be operating at a low resolution, and sending low resolution
images to the control system 14 via the network 20. The control
system 14 may he receiving the low resolution images, and sending
the low resolution images to the display device 16 via the display
module 26. The display device 16 may be receiving the low
resolution images, and displaying the low resolution images for
viewing by a person or other user.
[0045] The process 50 starts at block 52, where the person focuses
on a particular low resolution image which is displayed on the
display device 16. From block 52, the process advances to block 54,
where the eye tracking device 22 detects the position of a person's
eyes who is viewing the display device 16, and sends an indication
of the detected eye position to the eye tracking module 28.
[0046] From block 54, the process advances to block 56, where the
eye tracking module 28 associates the indication of the detected
eye position with a position on the display device 16, associates
the position on the display device 16 with an image on the display
device 16, and associates the image on the display device 16 with
an individual camera 12.
[0047] From block 56, the process advances to block 58, where the
computing device 24 determines if the person has been focusing on
the same image for a predetermined period of time. The computing
device 24 may determine if the person has been focusing on the same
image for a predetermined period of time in any suitable manner.
For example, according to various embodiments, the computing device
24 may maintain a timer which resets every time the eye tracking
module 28 associates the indication of the detected eye position
with a particular image on the display screen 16. According to
various embodiments, the predetermined period of time may be in the
range of approximately 1.5 to 2 seconds. According to other
embodiments, the predetermined period of time may be more than 2
seconds or less than 1.5 seconds.
[0048] At block 58, if the computing device 24 determines that the
person has been focusing on the same image for at least the
predetermined period of time, the process advances to block 60,
where the camera resolution module 30 generates a change of
resolution instruction and sends the change of resolution
instruction to the associated camera 12 (i.e., the camera 12
associated with the viewed image). The change of resolution
instruction is an instruction to change the resolution of the
associated camera 12 from the low resolution to a high
resolution.
[0049] From block 60, the process advances to block 62, where the
associated camera 12 receives the change of resolution instruction,
and the resolution selection module 18 causes the associated camera
12 to switch from the low resolution to the high resolution. From
block 62, the process advances to block 64, where the associated
camera 12 now captures a high resolution image and sends the high
resolution image to the control system 14 via the network 20.
[0050] From block 64, the process advances to block 66, where the
control system 14 receives the high resolution image from the
associated camera 12, and sends the high resolution image to the
display device 16 via the display modulo 26. From block 66, the
process advances to block 68, where the display device 16 receives
the high resolution image, and displays the high resolution image
for viewing by the person or other user. As described hereinabove,
the high resolution image may occupy a larger area of the display
device than any of the individual low resolution images.
[0051] From block 68, the process returns to block 52 when the
person changes his or her focus from the high resolution image to a
different image (e.g., a low resolution image) which is displayed
on the display device 16. The process described at blocks 52-68 may
be repeated any number of times.
[0052] The execution of the process described in blocks 52-68
results a low resolution image on the display device 16 being
replaced with a high resolution image after the person is focused
on the low resolution image for a predetermined period of time.
Thus, only one camera 12 at a time is sending high resolution
images, thereby minimizing the bandwidth needed to effectively
operate the system 10. For the four camera 12 example of the system
10 of FIG. 1, the utilization of the above-described method 50
lowers the needed bandwidth on the order of approximately 75% (from
38 Mbt/s to approximately 9.5 Mbt/s).
[0053] FIG. 6 illustrates various embodiments of another method 80
for controlling the data flow rate of a video network. The method
80 may be implemented by various components of the system 40 of
FIG. 4. For purposes of simplicity, the method will be described in
the context of the system 40 of FIG. 4.
[0054] Prior to the start of the process 80, each of the cameras 12
may be operating at a low resolution, and sending low resolution
images to the control system 14 via the network 20. The control
system 14 may be receiving the low resolution images, and sending
the low resolution images to the display device 16 via the display
module 26. The display device 16 may be receiving the low
resolution images, and displaying the low resolution images for
viewing by a person or other user.
[0055] The process 80 starts at block 82, where the person focuses
on a particular low resolution image which is displayed on the
display device 16. From block 82, the process advances to block 84,
where the eye tracking device 22 detects the position of a person's
eyes who is viewing the display device 16, and sends an indication
of the detected eye position to the eye tracking module 28.
[0056] From block 84, the process advances to block 86, where the
eye tracking module 28 associates the indication of the detected
eye position with a position on the display device 16, associates
the position on the display device 16 with an image on the display
device 16, and associates the image on the display device 16 with
an individual camera 12.
[0057] From block 86, the process advances to block 88, where the
computing device 24 determines if the person has been focusing on
the same image for a predetermined period of time. The computing
device 24 may determine if the person has been focusing on the same
image for a predetermined period of time in any suitable manner.
For example, according to various embodiments, the computing device
24 may maintain a timer which resets every time the eye tracking
module 28 associates the indication of the detected eye position
with a particular image on the display screen 16. According to
various embodiments, the predetermined period of time may be in the
range of approximately 1.5 to 2 seconds. According to other
embodiments, the predetermined period of time may be more than 2
seconds or less than 1.5 seconds.
[0058] At block 88, if the computing device 24 determines that the
person has been focusing on the same image for at least the
predetermined period of time, the process advances to block 90,
where the camera resolution module; 30 generates a change of
resolution instruction and sends the change of resolution
instruction to the associated camera 12 (i.e., the camera 12
associated with the viewed image). The change of resolution
instruction is an instruction to change the resolution of the
associated camera 12 from the low resolution to a high
resolution.
[0059] From block 90, the process advances to block 92, where the
associated camera 12 receives the change of resolution instruction,
and the resolution selection module 18 causes the associated camera
12 to switch from the low resolution to the high resolution. From
block 92, the process advances to block 94, where the associated
camera 12 now captures a high resolution image.
[0060] From block 94, the process advances to block 96, where the
image resolution module 42 of the associated camera 12 determines
if the high resolution image captured by the associated camera 12
includes a particular object of interest (e.g., a human face). From
block 96, the process advances to either block 98 or to block
108.
[0061] At block 96, if the image resolution module 42 determines
that the high resolution image includes the particular object of
interest, the process advances from block 96 to block 98, where the
image resolution module 42 defines a location of the object of
interest within the high resolution image (i.e., location
information), and generates two images associated with the high
resolution image. The first associated image is a high resolution
image of the object of interest portion of the high resolution
image captured by the associated camera 12. The second associated
image is a low resolution image of the high resolution image
captured by the associated camera 12. From block 98, the process
advances to block 100, where the associated camera 12 sends the
location information and the two associated images to the control
system 14 via the network 20 (collectively the composite
information).
[0062] From block 100, the process advances to block 102, where the
composite image module 44 receives the composite information and
generates a composite image based on the received composite
information. The composite image module 44 may generate a composite
image in any suitable manner. For example, according to various
embodiments, the composite image module 44 generates the composite
image by superimposing the first associated image on the second
associated image at the location defined by the image resolution
module 42.
[0063] According to other embodiments, the composite image module
44 generates the composite image by deleting a portion of the
second associated image corresponding to the location of the object
of interest as determined by the image resolution module 42. The
composite image module 44 then inserts the first associated image
onto the remaining portion of the second associated image at the
location previously occupied by the deleted portion of the second
associated image.
[0064] According to yet other embodiments, the composite image
module 44 generates the composite image by deleting a portion of
the second associated image corresponding to the location of the
object of interest as determined by the image resolution module 42.
The composite image module 44 then positions the remaining portion
of the second associated image over the first associated image such
that the location previously occupied by the deleted portion of the
second associated image is aligned with the first associated
image.
[0065] From block 102, the process advances to block 104, where the
control system 14 sends the composite image to the display device
16 via the display module 26. From block 104, the process advances
to block 106, where the display device 16 receives the composite
image, and displays the composite image for viewing by the person
or other user. The composite image may occupy a larger area of the
display device 16 than any of the individual low resolution
images.
[0066] From 106, the process returns to block 82 when the person
changes his or her focus from the composite image to a different
image (e.g., a low resolution image) which is displayed on the
display device 16. The process described at blocks 82-106 may be
repeated any number of times.
[0067] The execution of the process described in blocks 82-106
results in a low resolution image on the display device 16 being
replaced with a composite image after the person is focused on the
low resolution image for a predetermined period of time. Thus, only
one camera 12 at a time is sending high resolution images (e.g.,
the object of interest portion of the high resolution image
captured by the associated camera 12), thereby minimizing the
bandwidth needed to effectively operate the system 40. For the four
camera 12 example of the system 40 of FIG. 4, the utilization of
the above-described method 80 lowers the needed bandwidth on the
order of approximately 88% (from 38 Mbt/s to approximately 4.5
Mbt/s) when the high resolution image captured by the associated
camera 12 includes the object of interest.
[0068] At block 96, if the image resolution module 42 determines
that the high resolution image does not include the particular
object of interest, the process advances from block 96 to block
108, where the associated camera 12 sends the high resolution image
to the control system 14. From block 108, the process advances to
block 110, where the control system 14 sends the high resolution
image to the display device 16 via the display module 26. From
block 110, the process advances to block 112, where the display
device 16 receives the high resolution image, and displays the high
resolution image for viewing by the person or other user. As
described hereinabove, the high resolution image may occupy a
larger area of the display device 16 than any of the individual low
resolution image.
[0069] From block 112, the process returns to block 82 when the
person changes his or her focus from the high resolution image to a
different image (e.g., a low resolution image) which is displayed
on the display device 16. The process described at blocks 82-96 and
108-112 may be repeated any number of times.
[0070] The execution of the process described in blocks 82-96 and
108-112 results a low resolution image on the display device 16
being replaced with a high resolution image after the person is
focused on the low resolution image for a predetermined period of
time. Thus, only one camera 12 at a time is sending high resolution
images, thereby minimizing the bandwidth needed to effectively
operate the system 40. For the four camera 12 example of the system
40 of FIG. 4, the utilization of the above-described method 80
lowers the needed bandwidth on the order of approximately 75% (from
38 Mbt/s to approximately 9.5 Mbt/s) when the high resolution image
captured by the associated camera 12 does not include the object of
interest.
[0071] Nothing in the above description is meant to limit the
invention to any specific materials, geometry, or orientation of
elements. Many part/orientation substitutions are contemplated
within the scope of the invention and will be apparent to those
skilled in the art. The embodiments described herein were presented
by way of example only and should not be used to limit the scope of
the invention.
[0072] Although the invention has been described in terms of
particular embodiments in this application, one of ordinary skill
in the art, in light of the teachings herein, can generate
additional embodiments and modifications without departing from the
spirit of, or exceeding the scope of, the claimed invention.
Accordingly, it is understood that the drawings and the
descriptions herein are proffered only to facilitate comprehension
of the invention and should not be construed to limit the scope
thereof.
* * * * *